Papers with deep learning community
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent interpretability methods project weights and hidden states obtained from the forward pass to the models’ vocabularies, helping to uncover how information flows within LMs. |
| Approach: | They propose to cast a gradient matrix as a low-rank linear combination of forward and backward passes’ inputs and then to project these gradients into vocabulary items. |
| Outcome: | The proposed method can be cast as a low-rank linear combination of forward and backward passes’ inputs and project these gradients into vocabulary items. |
Translation vs. Dialogue: A Comparative Analysis of Sequence-to-Sequence Modeling (2020.coling-main)
Copied to clipboard
| Challenge: | Existing models for machine translation and dialogue response generation require a large number of handcrafted features. |
| Approach: | They propose to interpret a general neural model comparatively by using the seq2seq model in two mainstream NLP tasks. |
| Outcome: | The proposed model is used in two mainstream NLP tasks and is compared with a standard model. |